This disclosure relates generally to fraud scoring models, and more particularly to systems and methods for maintaining a pre-defined fixed score distribution by using online calibration of the fraud scores in an adaptive manner.
The distribution of model scores produced by a payment card fraud detection solution such as the Falcon™ Fraud Manager deployed by Fair Isaac Corporation, or any other scoring model, can change with time and vary among different clients. However, some customers would prefer the score distribution to remain constant in production to meet operational constraints in view of fixed analyst resources to review the cases. If the case volume is changing, other customers would like to be alerted to this shift in order to respond with additional resources.
The raw score can be calibrated based on the percentile of the initial score distribution, thereby maintaining a fixed distribution of the final calibrated scores. The percentile is the common key between the production raw score distribution and the reference curve produced from the model building exercise, and is typically included in a customer's model report. However, generating the percentiles of raw scores in production can be impractical, particularly for transaction models given both the time efficiency and fixed memory requirement for transaction-based online scoring models.
Conventional score calibration approaches are flawed in several ways. First, the score distribution characteristics, and hence related staffing requirements, are usually averaged over the previous 12-24 months of data to develop the model, and do not reflect the variation from month to month in case generation, for example during holiday periods where case volume dramatically increases given the change in customer behavior associated with the holiday. Also, the models using conventional calibration techniques cannot anticipate changes in the score distribution characteristics of the model based on changes in fraud rate or natural economic trends in spending behavior that will happen over the life of the model. The change in the fraud rate or economic trending will cause the performance characteristics of the model to not oscillate around an average behavior, but rather to trend differently altogether. However, it is known to be very undesirable to change case generation rules constantly by choosing different score thresholds in response to changing score distributions.